Semantic Services Grid in Flood-forecasting Simulations

Flooding in the major river basins of Central Europe is a recurrent event affecting many countries. Almost every year, it takes away lives and causes damage to infrastructure, agricultural and industrial production, and severely affects socio-economic development. Recurring floods of the magnitude and frequency observed in this region is a significant impediment, which requires rapid development of more flexible and effective flood-forecasting systems. In this paper we present design and development of the flood-forecasting system based on the Semantic Grid services. We will highlight the corresponding architecture, discovery and composition of services into workflows and semantic tools supporting the users in evaluating the results of the flood simulations. We will describe in detail the challenges of the flood-forecasting application and corresponding design and development of the service-oriented model, which is based on the well known Web Service Resource Framework (WSRF). Semantic descriptions of the WSRF services will be presented as well as the architecture, which exploits semantics in the discovery and composition of services. Further, we will demonstrate how experience management solutions can help in the process of service discovery and user support. The system provides a unique bottom-up approach in the Semantic Grids by combining the advances of semantic web services and grid architectures

Leveraging Interactivity and MPI for Environmental Applications

This paper describes two different approaches to exploiting interactivity and MPI support available in the Interactive European Grid project.The first application is an air pollution simulation using Lagrangian trajectory model to simulate the spread of pollutant particles released into the atmosphere. The performance of the sequential implementation of the application was not satisfactory, therefore a parallelization was planned. The MPI programming model was used because of some previous experience with it and its support in the grid infrastructure to be used. Then the interactivity enabling the user to receive visualizations of simulation steps and to exercise control over the application running in the grid was added. The user interface for interacting with the application was implemented as a plug-in into the Migrating Desktop user interface client platform. The other application is an interactive workflow management system, which is a modification of a previously developed system for management of applications composed of web and grid services. It allows users to manage more complex jobs, composed of several program executions, in an interactive and comfortable manner. The system uses the interactive channel of the project to forward commands from a GUI to the on-site workflow manager, and to control the job during execution. This tool is able to visualize the inner workflow of the application. User has complete in-execution control over the job, can see its partial results, and can even alter it while it is running. This allows not only to accommodate the job workflow to the data it produces, extend or shorten it, but also to interactively debug and tune the job

Architecture of a Function-as-a-Service Application

Serverless computing and Function-as-a-Service (FaaS) are programming paradigms that have many advantages for modern, distributed and highly modular applications. However, the process of transforming a legacy, monolithic application into a set of functions suitable for a FaaS environment can be a complex task. It may be questionable whether the obvious advantages received from such a transformation outweigh the effort and resources spent on it. In this paper we present our continuing research aimed at the transformation of legacy applications into the FaaS paradigm. Our test subject is an airport visibility system, a sub-class of the meteorological services required for airport operations. We have chosen to modularize the application, divide it into parts that can be implemented as functions in the FaaS paradigm, and provide it with a simple cloud-based data management layer. The tools that we are using are Apache OpenWhisk for FaaS and Airflow for workflow management, Apache Airflow for workflow management and NextCloud for cloud storage. Only a part of the original application has been transformed, but it already allows us to draw some conclusions and especially start forming a generalized picture of a Function-as-a-Service application

Application of Advanced Information and Communication Technologies in a Local Flood Warning System

This paper deals with the practical application of a local flood warning system. The system is built on the mathematical model of a selected area. The rainfall-runoff processes are simulated in real-time. The warning system is designed as an on-line, real-time data inputs-processing system so that it can provide a timely warning. The warning system is based on a mathematical model and it uses modern information and communication technology tools. For the system to work properly, it is absolutely necessary to adhere to a real mathematical model, and therefore a calibration on real historical data and direct measurements is required. This article describes the tasks of data collection, of building the mathematical model of the rainfall-runoff process, and the monitoring system design. The composed algorithm is able, based on the measured input data and the modeled situation, send a notification message to the monitoring centre and warn respective civil protection authorities via SMS messages

Integrated System for Hydraulic Simulations

The work described in this paper is aimed at applying and co-operating of modern information technologies and mathematical modeling to make a risk analysis of the water-supply in big cities. It is instrumental in the investigation of the hydraulics of water-supply systems using the simulation model EPANET executed on the underlying high-performance computing infrastructure. The simulation process is integrated with the GIS environment in order to correct input data and visualize the simulation output. Input data for the model can be modified directly within the designed scientific gateway which enables hydraulic domain experts to interact comfortably with the HPC capacity. Furthermore, the system includes some data mining capabilities forming bridges between the hydraulic data storage and available hydrological measurements focused on water consumption modeling and predictions. In simulating the main emphasis is given to optimize the measure of a similarity between the mathematical model and the real system in order to obtain reliable results

Collaborative Environment for Grid-based Flood Prediction

This paper presents the design, architecture and main implementation features of the flood prediction application of the Task 1.2 of the EU IST CROSSGRID. The paper begins with the description of the virtual organization of hydrometeorological experts, users, data providers and customers supported by the application. Then the architecture of the application is described, followed by used simulation models and modules of the collaborative environment. The paper ends with vision of future development of the application

USING ADVANCED DATA MINING AND INTEGRATION IN ENVIRONMENTAL PREDICTION SCENARIOS

We present one of the meteorological and hydrological experiments performed in the FP7 project ADMIRE. It serves as an experimental platform for hydrologists, and we have used it also as a testing platform for a suite of advanced data integration and data mining (DMI) tools, developed within ADMIRE. The idea of ADMIRE is to develop an advanced DMI platform accessible even to users who are not familiar with data mining techniques. To this end, we have designed a novel DMI architecture, supported by a set of software tools, managed by DMI process descriptions written in a specialized high-level DMI language called DISPEL, and controlled via several different user interfaces, each performing a different set of tasks and targeting different user group

PROCESS Data Infrastructure and Data Services

Due to energy limitation and high operational costs, it is likely that exascale computing will not be achieved by one or two datacentres but will require many more. A simple calculation, which aggregates the computation power of the 2017 Top500 supercomputers, can only reach 418 petaflops. Companies like Rescale, which claims 1.4 exaflops of peak computing power, describes its infrastructure as composed of 8 million servers spread across 30 datacentres. Any proposed solution to address exascale computing challenges has to take into consideration these facts and by design should aim to support the use of geographically distributed and likely independent datacentres. It should also consider, whenever possible, the co-allocation of the storage with the computation as it would take 3 years to transfer 1 exabyte on a dedicated 100 Gb Ethernet connection. This means we have to be smart about managing data more and more geographically dispersed and spread across different administrative domains. As the natural settings of the PROCESS project is to operate within the European Research Infrastructure and serve the European research communities facing exascale challenges, it is important that PROCESS architecture and solutions are well positioned within the European computing and data management landscape namely PRACE, EGI, and EUDAT. In this paper we propose a scalable and programmable data infrastructure that is easy to deploy and can be tuned to support various data-intensive scientific applications

Reference Exascale Architecture (Extended Version)

While political commitments for building exascale systems have been made, turning these systems into platforms for a wide range of exascale applications faces several technical, organisational and skills-related challenges. The key technical challenges are related to the availability of data. While the first exascale machines are likely to be built within a single site, the input data is in many cases impossible to store within a single site. Alongside handling of extreme-large amount of data, the exascale system has to process data from different sources, support accelerated computing, handle high volume of requests per day, minimize the size of data flows, and be extensible in terms of continuously increasing data as well as an increase in parallel requests being sent. These technical challenges are addressed by the general reference exascale architecture. It is divided into three main blocks: virtualization layer, distributed virtual file system, and manager of computing resources. Its main property is modularity which is achieved by containerization at two levels: 1) application containers - containerization of scientific workflows, 2) micro-infrastructure - containerization of extreme-large data service-oriented infrastructure. The paper also presents an instantiation of the reference architecture - the architecture of the PROCESS project (PROviding Computing solutions for ExaScale ChallengeS) and discusses its relation to the reference exascale architecture. The PROCESS architecture has been used as an exascale platform within various exascale pilot applications. This paper also presents performance modelling of exascale platform with its validation